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FINAL  REPORT  ON  CONTRACT  DAAG29-80-C-0036 


This  is  the  final  report  on  Contract  DAAG29-80-C-0036  entitled 
"Multiple  Decision  Ranking  and  Selection  Procedures."  The  report  covers 
the  period  November  1,  1979  through  July  31,  1981.  The  present  contract 
with  a  budget  of  $20,000  was  originally  funded  for  the  period  November  1, 
1979  through  October  31,  1980;  a  no-new-funds  extension  was  granted  to 
continue  research  through  July  31,  1981.  The  contract  is  a  follow-up  to 
Contract  DAAG29-77-C-0003  (and  earlier  ones)  which  had  similar  missions. 


Because  of  the  closeness  of  the  objectives  of  these  contracts,  this  report 
lists  some  technical  reports  and  papers,  the  research  for  which  was  initated 
or  completed  under  the  earlier  contract,  but  which  was  not  completed  or 
published  until  after  the  final  report  for  that  contract  was  submitted. 

The  following  pages  list  the  technical  reports  and  published  papers 
(and  those  accepted  for  or  submitted  for  publication)  which  were  written 
with  the  full  or  partial  support  of  these  contracts.  The  contents  of  most 
of  these  reports  and  papers  have  already  been  described  in  detail  in  the 
three  Semi-Annual  Progress  Reports  covering  the  periods  August  16,  1978  - 
December  31,  1979,  January  1,  1980  -  June  30,  1980,  and  July  1,  1980  - 

December  31,  1980.  >t  A  *  f 
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■  Most  of- the  papers 'deal  with  research  on  the  subject  of  "Statistical 
multiple-decision  ranking  and  selection  procedures,"  a  statistical 
methodology  pioneered  by  the  Principal  Investigator  and  several  colleagues; 
some  of  the  papers  deal  with  related  statistical  techniques.  Interest  in 
this  subject  continues  to  grow  as  is  evidenced  by  the  large  number  of 
published  papers  in  this  area,  and  by  the  appearance  in  the  last  few  years 
of  three  textbooks  devoted  exclusively  to  this  subject.  The  books  are: 
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Selecting  and  Ordering  Populations:  A  New  Statistical 
Methodology  (Wiley,  1977)  by  J.  Gibbons,  I.  Olkin  and  M.  Sobel, 

( 569  pages ) . 

Multiple  Decision  Procedures :  Theory  and  Methodology  of 
Selecting  and  Ranking  Populations  (Wiley,  1979)  by  S.S.  Gupta 
and  S.  Panchapakesan ,  (573  pages). 

Nonparametric  Sequential  Selection  Procedures  (BirkhaUser,  1980) 
by  H.  Buringer,  H.  Martin  and  K.  -  H.  Schriever,  (488  pages). 

An  invited  review  of  the  Gibbons-Olkin  Sobel  book  by  the  Principal 
Investigator  appeared  in  the  Journal  of  the  American  Statistical  Associa¬ 
tion,  75  (1980),  751-756. 

The  Principal  Investigator,  along  with  J.  Gibbons  (University  of 
Alabama),  S.S.  Gupta  (Purdue  University)  and  I.  Olkin  (Stanford  University), 
organized  and  presented  the  1979  Annual  Meeting  Short  Course  on  the 
subject  of  "Selecting  and  ordering  populations"  for  the  American  Statis¬ 
tical  Association.  The  course  was  held  August  11  -  12,  1979  in  Washington, 

D . C . ;  videotapes  of  the  course  should  be  available  shortly  from  the  ASA . 

The  course  was  very  well  attended,  and  the  topics  aroused  considerable 
interest  on  the  part  of  the  participants.  A  significant  portion  of  the 
statistical  methodologies  which  were  presented  and  discussed  represented 
the  results  of  research  by  the  Principal  Investigator  and  his  colleagues 
at  Cornell  University  and  elsewhere. 

In  addition,  the  Principal  Investigator  with  Professor  Shanti  Gupta 
offered  a  5-day  seminar  on  "Selection  and  Ranking  Procedures"  sponsored  by 
The  George  Washington  University,  School  of  Engineering  and  Applied  Science 
and  AMK  Berlin  in  Berlin,  Germany,  November  17  -  21,  1980. 


RECENT  RESEARCH  ON  CONTRACT  DAAG29-80-C-0036 


The  main  thrust  of  new  research  on  the  present  contract  stems  from  a 
selection  problem  associated  with  quantal  response  curves;  this  problem 
has  considerable  practical  importance.  The  statistical  model  underlying 
the  quantal  response  curve  problem  was  described  in  detail  in  the  Semi-Annual 
Progress  Report  on  this  contract  which  covered  the  period  January  1,  1980  - 
June  30,  1980.  It  was  pointed  out  there  that  if  certain  assumptions  hold 
for  the  k  >  2  quantal  response  curves,  then  the  selection  problem  for  the 
k  quantal  response  curves  can  be  solved  by  solving  an  associated  selection 
problem  for  k  >  2  Bernoulli  populations. 

Recognition  of  the  fact  that  a  solution  of  the  Bernoulli  selection 
problem  leads  (under  appropriate  assumptions)  to  a  solution  of  the  quantal 
response  selection  problem  suggested  that  intensive  study  of  the  former 
problem  would  be  profitable.  Such  an  investigation  was  undertaken  by  the 
Principal  Investigator  along  with  one  of  his  Ph.D.  students,  Ms.  Radhika 
Kulkarni.  This  study  of  the  Bernoulli  selection  problem  led  to  a  major 
research  breakthrough .  Their  findings  are  not  only  relevant  to  the 
quantile  selection  problem,  but  are  also  very  important  in  their  own  right. 

The  first  technical  report  on  this  subject,  TR  510  entitled  "Closed 
adaptive  sequential  procedures  for  selecting  the  best  of  k  >  2  Bernoulli 
populations"  has  just  been  completed.  Some  of  the  results  discussed  in  that 
report  were  presented  at  the  Third  Symposium  on  Statistical  Decision 
Theory  and  Related  Topics  held  at  Purdue  University,  June  1-5,  1981  and 
will  be  published  in  the  Proceedings  of  the  Symposium.  (Earlier  presen¬ 
tations  of  portions  of  the  paper  were  given  at  the  spring  regional  meeting 
of  the  Biometric  Society  and  American  Statistical  Association  in  Richmond, 
Virginia,  March  23,  1981,  and  at  the  Annual  Meeting  of  the  Israeli 


Statistical  Society  in  Tel  Aviv,  Israel  on  May  28,  1981.)  An  abstract  of 
the  paper  is  given  below.  Additional  papers  on  this  subject  are  in  prep- 
ation.  A  summary  of  some  of  the  findings  was  also  presented  in  a  seminar 
at  the  Ballistic  Research  Laboratories,  Aberdeen,  on  June  12,  1981.  It  is 
hoped  that  the  studies  that  are  in  progress  will  form  the  basis  for 
recommendations  for  new  generalized  sampling  plans  that  might  be  incor¬ 
porated  in  a  Military  Standard  testing  procedure  for  comparative  Bernoulli 
trials.  Also,  as  pointed  out  above,  the  results  will  be  applicable  to 
the  quantal  response  curve  selection  problem. 


Abstract  of  TR  510  (July  1981)  entitled  "Closed  adaptive  sequential 
procedures  for  selecting  the  best  of  k  >  2  Bernoulli  populations"  by 
R.E.  Bechhofer  and  R.V.  Kulkarni. 


"The  goal  of  selecting  that  one  of  k  >  2  Bernoulli  populations 
which  has  the  largest  single-trial  "success"  probability  p^  =  max  (p^, . . .  ,p^} 
is  treated.  Consideration  is  restricted  to  procedures  which  take  no  more 
than  n  observations  from  any  one  of  the  k  populations.  One  such  pro¬ 

cedure  is  the  single-stage  procedure  of  Sobel  and  Huyett  [1957]  which  takes 
exactly  n  observations  from  each  of  the  k  populations.  We  propose  a 

one’-at-a-time  adaptive  sampling  rule  (R*)  which  when  used  in  conjunction 
with  a  particular  stopping  rule  (5*)  and  terminal  decision  rule  (T*) 
achieves  the  same  probability  of  a  correct  selection  as  does  the  single- 

stage  procedure  uniformly  in  p  =  (p_ > • . • »P.  )•  Letting  N  denote  the 

%  i.  K 

random  total  number  of  observations  to  terminate  sampling  using  the  procedure 

(R*,S*,T*)  we  show  that  n  <  N  <  kn-1;  for  P^j  -►  0  we  have 

P{N  =  kn-1  |  p)  -*■  1  while  for  p,.,  +  1  we  have  P{N  =  n  |  p}  1.  For 

LJ- J  ^ 

k  =  2  the  sampling  rule  R*  (the  conjugate  sampling  rule  7*)  which  is 
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stationary  is  optimal  in  the  sense  that  it  minimizes  ECNfCp^tp^)) 
uniformly  in  <p1$p2)  for  p^^  +  p2  >  1  (p^  +  p2  <  1)  among  all  sampling 
rules  which  use  (S*,T*)  and  which  take  no  more  than  n  observations 
from  either  population;  R*  has  additional  optimal  properties  for  k  =  2. 

The  procedure  (R*,S*,T*)  is  generalized  for  k  >  2  to  accommodate  such 
goals  as  "Selecting  the  s  (1  <  s  <  k-1)  "best"  Bernoulli  populations 
with  regard  to  order,"  and  is  shown  to  have  desirable  properties  for  these 
goals  as  well.  Some  conjectures  are  made  concerning  the  optimality  of 
(R *,S*,T*)  for  k  >  2.  The  performance  of  (R*,S*,T*)  is  compared  for 
k  >  2  with  that  of  other  sequential  selection  procedures  that  have  been 
proposed  in  the  literature .  An  extensive  bibliography  is  included. 

The  Principal  Investigator  has  submitted  a  research  proposal  to 
ARO-D  to  pursue  new  investigations  in  this  and  related  areas,  and  to 
further  exploit  research  findings  already  obtained.  At  this  time  the 
Principal  Investigator  would  like  to  express  his  thanks  to  the  Army  Research 
Office  -  Durham  for  its  strong  support  of  this  research  activity.  He  is 
indebted  to  Mr.  Ralph  E.  Shear  of  the  BRL  for  having  brought  to  his  attention 
the  interesting  and  important  practical  problem  that  led  to  the  present 
studies,  and  to  Dr.  Robert  Launer  of  ARO-D  for  his  constant  encouragement. 
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Summary  of  Research  Completed  on  the  Contract 

Technical  reports,  the  research  for  which  was  supported  in  whole  or  in 
part  by  ARO  Contract  DAAG29-77-C-0003. 

(These  reports  were  completed  after  the  termination  of  DAAG29-77-C-0003, 
but  before  the  start  of  DAAG29-80-C-0036 .  Although  they  were  credited  to 
the  former  contract,  they  are  not  listed  in  the  FINAL  REPORT  on  that  con¬ 
tract  dated  September  20,  1978). 


Bechhofer,  R.E.  and  Santner,  T.J.:  "A  note  on  the  lower  bound  for  the 
P{CS)  of  Gupta's  subset  selection  procedures,"  TR  401, 

January  1979. 

Bechhofer,  R.E.  and  Tamhane,  A.C. :  "Incomplete  block  designs  for  comparing 
treatments  with  a  control  (I):  general  theory,"  TR  414,  March  1979. 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  com¬ 
paring  treatments  with  a  control  (II):  optimal  designs  for 
p  =  2(1)6,  k  =  2  and  p  =  3,  k  =  3,"  TR  425,  May  1979. 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  com¬ 
paring  treatments  with  a  control  (III):  optimal  designs  for 
p  =  4,  k  =  3  and  p=5,k=3,"  TR  436,  October  1979. 

Technical  reports,  the  research  for  which  was  supported  in  whole  or  in 

part  by  ARO  Contract  DAAG29-80-C-0036 


Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  com¬ 
paring  treatments  with  a  control  (IV):  optimal  designs  for  p  =  4, 
k  =  4,"  TR  440,  January  1980. 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  com¬ 
paring  treatments  with  a  control  (V):  optimal  designs  for  p  =  6, 
k  =  3,"  TR  441,  June  1980. 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  com¬ 
paring  treatments  with  a  control  (VI);  conjectured  minimal  complete 
class  of  generator  designs  for  p  =  5,  k  =  4  and  p  =  6,  k  =  4," 

TR  453,  April  1980. 

McCulloch,  C.E.:  "Conditions  under  which  E{N..}  =  »  for  Tong's  adaptive 
solution  to  ranking  and  selection  problems,"  TR  480,  September  1980. 
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Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Tables  of  optimal  allocation  of 
observations  for  comparing  treatments  with  a  control,"  TR  489, 
January  1981. 

Bechhofer,  R.E.  and  Dunnett,  C.W.:  "Multiple  comparisons  for  orthogonal 
contrasts:  examples  and  tables,"  TR  495,  March  1981. 

Bechhofer,  R.E.  and  Kulkarni,  R.V.:  Closed  adaptive  sequential  procedures 
for  selecting  the  best  of  k  >  2  Bernoulli  populations,"  TR  510, 
July  1981. 

Faltin,  F.W.  and  McCulloch,  C.E.:  "On  the  small-sample  properties  of  the 
Olkin-Sobel-Tong  estimator  of  the  probability  of  correct  selection," 
Florida  State  University  statistics  report  M-581,  July  1981  (based  on 
research  done  while  the  authors  were  at  Cornell  University). 


Papers  (published  or  accepted  for  publication) 


Barton,  R.R.  and  Turnbull,  B.W.:  "A  survey  of  covariance  models  for 
censored  life  data  with  an  application  to  recidivism  analysis," 
Communications  in  Statistics  -  -  Theory  and  Methods,  A8  (8),  1979, 
723-750. 

Bechhofer,  R.E. :  Invited  review  of  Selecting  and  Ordering  Populations  -  - 
A  New  Statistical  Methodology  by  J.  Gibbons,  I.  Olkin  and  M.  Sobel 
(John  Wiley  1977),  Journal  of  the  American  Statistical  Association, 

75  (1980),  751-756. 

Bechhofer,  R.E.  and  Kulkarni,  R.V. :  "Closed  adaptive  sequential  procedures 
for  selecting  the  best  of  k  >  2  Bernoulli  populations."  To 
appear  in  the  Proceedings  of  the  Third  Purdue  Symposium  on 
Statistical  Decision  Theory,  held  at  Purdue  University,  June  1-5, 

1981. 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  com¬ 
paring  treatments  with  a  control:  general  theory."  Technometrics, 

23  (1981),  45-57. 

Bechhofer,  R.E.  and  Tamhane,  A.C.:  "Incomplete  block  designs  for  com¬ 
paring  treatments  with  a  control  (II):  optimal  designs  for  p  =  2(1)6, 
k  =  2  and  p  =  3,  k  =  3."  Accepted  for  publication  in  Sankhya,  B. 

Faltin,  F.W.:  "Performance  of  the  Sobel-Tong  estimator  of  the  probability 
of  correct  selection  achieved  by  Bechhofer 's  single-stage  procedure 
for  the  normal  means  problem,"  Abstract  80t-59,  Bulletin  of  the 
Institute  of  Mathematical  Statistics,  9  (1980),  180. 

Faltin,  F.W. :  "A  quantile  unbiased  estimator  of  the  probability  of  correct 
selection  achieved  by  Bechhofer* s  single-stage  procedure  for  the 
two  population  normal  means  problem,  Abstract  80t-60,  Bulletin  of 
the  Institute  of  Mathematical  Statistics,  9  (1980),  180-181. 

Hooper,  J.H.  and  Santner,  T.J.:  "Design  of  experiments  for  selection  from 
ordered  families  of  distributions,"  Annals  of  Statistics.  7  (1979), 
615-643. 

McCulloch,  E.E.:  "Conditions  under  which  E{N  )  =  »  for  Tong's  adaptive 
solution  to  ranking  and  selection  problems."  Accepted  for  publication 
in  Communications  in  Statistics  -  -  Theory  and  Methods . 
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Santner,  T. J. :  "Designing  two-factor  experiments  for  selecting  inter¬ 
actions,"  Journal  of  Statistical  Planning  and  Inference  5  (1981), 
45-55." 


Santner,  T.J.  and  Snell,  M.J.:  "Small-sample  confidence  intervals  for 

p^  -  p2  and  P1/?2  2  x  2  contingency  tables,"  Journal  of  the 

American  Statistical  Association,  75  (1980),  386-394. 

Tamhane,  A.C.  and  Bechhofer,  R.E.:  "A  two-stage  rainimax  procedure  with 
screening  for  selecting  the  largest  normal  mean  (II):  an  improved 
PCS  lower  bound  and  associated  tables,"  Communications  in  Statistics 
Theory  and  Methods  A8  (4),  1979,  337-358. 
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Papers  submitted  for  publication 


Bechhofer,  R.E.,  and  Dunnett,  C.W.:  "Multiple  comparisons  for  orthogonal 
contrasts:  examples  and  tables," 

Bechhofer,  R.E.,  and  Tamhane,  A.C.:  "Tables  of  optimal  allocations  of 
observations  for  comparing  treatments  with  a  control." 

Faltin,  F.W.  and  McCulloch,  C.E.:  "On  the  small  sample  properties  of  the 
Olkin-Sobel-Tong  estimator  of  the  probability  of  correct  selection." 

Graduate  students  supported  by  Contract  DAAG29-80-C-0036 
Fred  W.  Faltin  Radhika  V.  Kulkarni 

Consultant  supported  by  Contract  DAAG29-80-C-0036 
A jit  C.  Tamhane 

Ph.D.  dissertation  and  M.S.  thesis,  the  research  for  which  was  supported 

in  whole  or  in  part  by  ARO  Contract  DAAG29-80-C-0036. 

Faltin,  Frederick  W. :  (M.S.  thesis)  "Estimating  the  probability  of 

correct  selection  achieved  by  Bechhofer' s  single-stage  procedure 
for  the  normal  means  problem."  May  1980. 

Kulkarni,  Radhika  V,:  (Ph.D.  dissertation)  "Closed  adaptive  sequential 
procedures  for  selecting  the  best  of  k  >  2  Bernoulli  populations." 
August  1981. 
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